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Abstract Body. 


Background / Context: 

Adolescents in the United States and their teachers face an enormous academic challenge with 
respect to reading comprehension. College and career readiness standards outlined in the 
Common Core (2012) place increased emphasis on preparing students to read increasingly 
complex text across a range of disciplinary content areas. At issue is how to develop the 
necessary skills and understandings to read the texts required of college and literacy-demanding 
occupations when fewer than 35% of students in secondary grades read proficiently (U.S. 
Department of Education, 2012, NCES 2012-457). Compounding the challenge is the reality 
that secondary-level courses are largely focused on disciplinary content where reading skills are 
assumed prerequisites and not an instructional priority (Kamil, Borman, Dole, Krai, Salinger, & 
Torgesen, 2008; Kennedy & Ihle, 2012). In addition, students must be able to proficiently read 
and understand a variety of complex text across several core content areas. Thus, in addition to 
the complex vocabulary, phrasing, and sentence and text structure found in secondary texts, 
adolescent students must deal with differences in how the vocabulary and text structures are used 
in various disciplines (Carnegie Council on Advancing Adolescent Literacy, 2010; Heller & 
Greenleaf, 2007; Lee & Spratley, 2010). The increased demands to read and learn from complex 
text and the reading proficiency levels of today’s adolescents bring into sharp relief the academic 
chasm that exists in secondary classes (Eason, Goldbert, Young, Geist, & Cutting, 2012) and the 
need to engage content-area teachers in the solution. 

Recognizing that secondary students are unlikely to meet the intentionally high goals of the CCS 
without improving outcomes in both reading comprehension and content learning, we sought to 
develop and implement an intervention that would align with content learning, improve reading 
comprehension, and be feasible for use by content area teachers in secondary schools. We used 
design experiment methodologies to better understand the contexts of secondary content-area 
instruction and the potential of teacher-directed and student-regulated English language arts and 
social studies interventions in middle and high school contexts. Our goal was to study the 
instructional contexts and align our practices with the realities of classrooms. To understand the 
instructional and disciplinary context of English language arts and social studies classrooms, we 
spent several months working with middle and high school teachers nominated by district 
administrators as experts. 

Purpose / Objective / Research Question / Focus of Study: 

Design experiments were conducted to examine the potential and feasibility of two interventions 
to improve student content knowledge and reading comprehension: Team-based Learning in 
social studies and Critical Reading Practices (teacher-directed and student-regulated 
comprehension practices) in English language arts. Both studies examined common research 
questions including (a) the potential and feasibility of two interventions, (b) intervention 
refinements, and (c) receptivity and perceptions of users. In this session we will present lessons 
learned and how design experiments shaped instructional procedures and practices. 

Setting: 

The English language arts and social studies design experiments were both conducted in 
classrooms in three school districts in two states, representing rural, suburban, and small urban 
areas. 
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Population / Participants / Subjects: 

Twelve soeial studies and seven English language arts teachers participated in the design studies. 
Teachers were nominated by their principal as expert teachers in their held. In order to be 
nominated, teachers were required to have at least three years of teaching experience, high 
ranking on evaluations, deep knowledge of content, and a willingness to commit to the study. 

Intervention / Program / Practice: 

A design experiment methodology was used to dehne and rehne interventions through multiple 
iterations. Because this was a prototype building process, intervention components were 
introduced over time and implemented in 1-2 week installments. Following we describe the 
essential components of the English language arts and social studies intervention. 

Critical Reading Practices was developed for English language arts and consisted of three 
primary phases. Phase 1 Text Set-Up was teacher-directed and designed to build and activate 
students’ background knowledge of concepts, vocabulary, text structure, and text features. Phase 
2, Text Analysis, was initially teacher-directed then transitioned to student regulated activities. In 
this phase students learned and applied comprehension monitoring techniques and engaged in 
text-based discussions with peers to extend and clarify understanding. In Phase 3, Text 
Synthesis, students worked with partners to summarize major events and Endings from text and 
record on graphic organizers. Recorded information was then used to prepare for quizzes, written 
responses, or classwide discussions. 

Based on previous reading and content learning approaches (Gersten, Baker, Smith-Johnson, & 
Domino, 2006; Vaughn et al., 2009), we identified content related instructional foci for social 
studies: (a) an overarching issue or question that served as a comprehension canopy for the 
instructional unit, (b) essential words or key vocabulary related to the unit, (c) a springboard or 
motivating video or idea that related to the unit, (d) appropriate text based instruction and 
reading, and (e) team based learning to promote text based discussions and evidence building. 

Research Design: 

We conducted a series of iterative design experiments across the school year with the teachers in 
each content area (English language arts and social studies) in order to analyze feasibility and 
teacher and student use of each of the elements of the intervention, refining implementation of 
the elements during each iteration (Cobb, Confrey, diSessa, Lehrer, & Schauble, 2003). Figure 1 
(see Appendix B) illustrates the process used. In each content area there were 3 iterative cycles 
of teacher implementation and feedback as a means for determining feasibility and use. 

Following refinements a 4**^ cycle was conducted with implementation of the full, refined 
intervention. 

Prior to the initial cycle of development and implementation, researchers developed an 
intervention prototype for building student independence in critical reading and improving 
content acquisition. The materials were developed based on previous work completed by the 
research team as well as observations of typical instruction in the classrooms. Teacher meetings 
were held generally one week prior to implementation of the prototype materials. A total of four 
all-day teacher meetings were held. At the initial meeting, teachers were asked about their 
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typical practice and to identify barriers to adoleseents’ reading comprehension. At following 
meetings, the first bloek of time was devoted to teaeher feedback on previously implemented 
praetices. Next, researchers shared prototype lesson plans and student materials to implement the 
praetiees for 1-2 weeks. Instruction in the strategies was provided if it was deemed neeessary by 
researeh personnel. Changes and modifications based on previous feedback were highlighted. 
Researeher personnel and teachers then applied the prototype strategies and materials to text and 
content the teaehers would be teaching during the next round of implementation. Lastly, 
teachers suggested additional modifieations to the materials. After implementation, teaehers 
partieipated in a foeus group in which they provided speeific feedback on the materials 
implemented. They also suggested modifications for future intervention versions. 

Data Collection and Analysis: 

Data sourees were at the teaeher level and consisted of lesson feedbaek forms, foeus groups, and 
in-classroom observations. Lesson feedbaek forms were completed by teaehers following each 
implementation. All intervention components were listed and teachers rated their level of 
implementation as “very effective”, “somewhat effective”, “not effeetive”, or “did not implement 
today. Teaeher foeus groups were also held following eaeh round of implementation. Foeus 
groups were approximately one hour each and were audio reeorded. Teachers responded to 
questions sueh as: (1) Briefly deseribe your experiences implementing the module; (2) How are 
the aetivities in the module similar to activities you typically use in the classroom? In what ways 
are they different?; (3) How did the planning time for this module eompare to “typical” lesson 
planning time? (4) What was the impact from implementing this module on the following: 
student partieipation? quality of student work? 

Teaehers were observed in person at least once per implementation cyele. In addition, audio 
recordings of instruction during implementation were collected. Observations were conducted 
during one class period (approximately 50 minutes) of implementation. In addition, observations 
of teaehers’ typical instructional practices were conducted at least 8 times during the sehool year. 
Observations focused on instruction and components of the intervention implemented, the 
amount of time implemented, and if the activity was connected to text. Other information 
recorded ineluded: type of text, type of reading (i.e., teacher read aloud, whole group, small 
group, partner, individual, and audio reeorded) and number of minutes, teehnology used (i.e., 
power point, interactive blackboard, document camera, audio or video elips), and a rating of 
student engagement (i.e., few, some, many, most). 

Qualitative Analysis . All data sourees (e.g., audio reeordings, field notes, notes of focus groups) 
were analyzed to determine whieh instructional methods and materials were most feasible and 
effective in the middle and high school classrooms. Using information sources from the first 
round of implementation, data was scanned for eommon eategories of phenomena and themes 
were developed based on this information. These themes were then used to analyze data at eaeh 
implementation data point, and refine materials for the following cycles of implementation. 

Quantitative Analysis. Data from in-person observations, lesson feedback forms, and audio 
taped observations was summarized and reported. This deseriptive data includes frequency 
counts and averages. 
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Findings / Results: 


English Language Arts Findings. Findings from the design experiments and teaeher input 
provided extensive information regarding intervention prototype refinements. Several important 
recommendations emerged regarding changes to prototype interventions including (a) routines 
that transfer to novels and informational text, (b) increased focus on comprehension fix -up 
strategies, (c) lessons that explicitly teach inference-making, (d) several options for vocabulary 
instruction, (e) lessen emphasis on classwide discussion and increase mini-discussions during 
reading, (f) incorporate more extensive writing assignments, (g) increase emphasis on 
documenting and using textual evidence. 

Social Studies Findings. Findings from the design experiments and observations of typical 
practice identified key areas for intervention refinement in the social studies classes including, 

(a) expansion of vocabulary instruction in the intervention to go beyond simple definitions which 
were the typical practice for teachers in 51% of the observations, (b) increased supported text 
reading and strategies for learning content from text (text was used in typical practice only 10% 
of observed time), (c) expanding text comprehension instruction beyond oral questioning (used 
76% of the time in typical practice), (d) increasing active student engagement in using text, 
discussion and problem solving about content, and identifying text evidence to support 
arguments, answers, and opinions, (e) providing several high quality, complex text options for 
instruction, (f) identifying multiple examples of broad, essential content related questions to 
elicit quality student discussion and deep processing of content and readings, (g) structured 
student discussion that includes student recording of key points and evidence. 

Conclusions: 

The design experiment methodologies provided extensive and important feedback into how to 
design and refine interventions. The teacher level data collection assisted in key refinements to 
the intervention to allow for increased fidelity and feasibility of implementation in ways that 
addressed both content learning and reading for understanding. The design experiment 
methodologies provided extensive and important feedback into how to design and refine 
interventions. We learned that collaborations with teachers were important but nonetheless 
challenging as there were diverse positions on what should be taught and practices that could be 
used. In the English language arts design experiments we learned there must be a base routine 
that can be adapted for different text genres. In addition, comprehension practices must be 
flexible and adaptable as there is no standard scope and sequence in which text types are 
introduced. In social studies, efficient practices and scaffolds were required for teachers to 
successfully facilitate reading of complex text within their content instruction and with a wide 
range of reading abilities Our observations led us to believe that more work was needed to 
structure successful peer-mediated activities and discussions. Few teachers were prepared to 
relinquish significant amounts of instructional time to student/partner-regulated strategies. We 
learned that developing an intervention that maps onto existing curriculum requires navigating 
and orchestrating may factors including differences by grades, heterogeneity in classrooms, and 
teacher preferences. 
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Appendix B Tables and Figures 

Figure 1 . Iterative Design Cyele 
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